Display device and method for operating same

ABSTRACT

A display device according to the present disclosure may comprise: a display which displays a content; and a controller which, upon receiving a voice command, controls the display to display content-related information corresponding to a scene at the time of receiving the voice command, wherein the controller may change arrangement of the content-related information on the basis of a category requested by the voice command.

TECHNICAL FIELD

The present disclosure relates to a display device and a method foroperating the same.

BACKGROUND ART

Recently, a digital TV service using a wired or wireless communicationnetwork has come into wide use. The digital TV service may providevarious services which cannot be provided by an existing analogbroadcast service.

For example, an Internet protocol television (IPTV) service or a smartTV service which is a digital TV service provides interactivity forenabling a user to actively select a type of a program to be viewed, aviewing time, etc. The IPTV service or the smart TV service may providevarious additional services, e.g., Internet search, home shopping,online games, etc. based on such interactivity.

When a user views the content, the user may have questions aboutproducts, characters, locations, etc. appearing in the content. In thiscase, it is inconvenient for the user to manually search for products,characters, locations, etc. appearing in the content by using the

Internet or the like. Also, it may be difficult for a user to searchproducts, characters, or locations that matches the one appearing in thecontent, even if the user searches for products, characters, locations,etc. appearing in the content one by one.

Thus, there may be a need for a method for automatically providinginformation about a product, characters, or location related to thecontent that is currently viewing by the user. Particularly, when thereis a user's request for content-related information, a method for moreaccurately providing information meeting the user's request may berequired.

DISCLOSURE OF THE INVENTION Technical Problem

An object of the present disclosure is to provide a display device thatprovides information corresponding to a user's request more quickly andpreferentially when providing content-related information, and anoperating method thereof.

An object of the present disclosure is to provide a display device thatmore accurately provides information corresponding to a user's requestwhen providing content-related information, and an operating methodthereof.

Technical Solution

A display device according to the present disclosure includes a displayconfigured to display a content and a controller configured to controlthe display to display content-related information corresponding to ascene at a time point, at which a voice command is received, when thevoice command is received, wherein the controller may change anarrangement of the content-related information on the basis of acategory requested by the voice command.

The controller may be configured to change the arrangement of thecontent-related information so that the information corresponding to therequested category is displayed in priority order rather than otherinformation.

When the content-related information comprises information correspondingto a first category and information corresponding to a second category,if the requested category is the first category, the display may beconfigured to display the information corresponding to the firstcategory in priority order rather than the information corresponding tothe second category.

When the information corresponding to the requested category does notexist in the content-related information, the controller may beconfigured to control the display so that other information other thanthe information corresponding to the requested category are displayed.

When the information corresponding to the requested category does notexist in the content-related information, the controller may beconfigured to determine whether information corresponding to othercategories other than the requested category exist in thecontent-related information.

When information corresponding to other categories other than therequested category exists in the content-related information, thecontroller may be configured to control the display so that theinformation corresponding to the other categories is displayed.

When the information corresponding to the requested category does notexist in the content-related information, the controller may beconfigured to determine whether the information corresponding to therequested category exists on a previous scene before receiving the voicecommand.

When the information corresponding to the requested category exists onthe scene before receiving the voice command, the controller may beconfigured to control the display so that the information correspondingto the requested category comprised in the previous scene is displayed.

When the information corresponding to the requested category does notexist in the content-related information, the controller may beconfigured to control the display so that all of the content-relatedinformation comprised in the content is displayed.

When the voice command for requesting the content-related information isreceived, the controller may be configured to acquire thecontent-related information that is metadata including a category.

The category may include at least one of a location, a product, acharacter, and others.

The content-related information may include categories corresponding toa frame time for each of a plurality of frames constituting the content.

When the content-related information is displayed, the controller may beconfigured to control the display so that an image captured at a timepoint, at which the voice command is received, is further displayed.

The controller may be configured to acquire the requested category basedon the voice command.

The controller may be configured to determine an order in which thecontent-related information is displayed based on the requestedcategory.

Advantageous Effects

According to the present disclosure, since the display device displaysthe information corresponding to the category requested by the user inthe priority order, there may be the advantage in that it is possible toprovide the user's convenience to quickly acquire the desiredinformation.

According to the present disclosure, if the information corresponding tothe category requested by the user does not exist, since the displaydevice displays the information of another category or the informationcorresponding to the requested category contained in the previous scene,there may be the advantage of more accurately providing the informationdesired by the user.

According to the present disclosure, since the display device displaysthe captured image together when displaying the content-relatedinformation, there may be the advantage that the user easily checkswhich scene the displayed content-related information is related to thecontent-related information.

According to the present disclosure, there may be an advantage in thatthe display device uses the content-related information that is themetadata including the category to more accurately provide theinformation corresponding to the category.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a displaydevice according to an embodiment of the present disclosure.

FIG. 2 is a block diagram illustrating a remote control device accordingto an embodiment of the present disclosure.

FIG. 3 is a view illustrating an actual configuration of the remotecontrol device according to an embodiment of the present disclosure.

FIG. 4 is a view illustrating an example of utilizing a remote controldevice according to an embodiment of the present disclosure.

FIG. 5 is a flowchart illustrating a method for operating a displaydevice according to an embodiment of the present disclosure.

FIG. 6 is a schematic view illustrating an example of a method foracquiring a required category and content-related information through adisplay device according to an embodiment of the present disclosure.

FIG. 7 is a view illustrating an example of a method for displayingcontent-related information for each category through the display deviceaccording to an embodiment of the present disclosure.

FIG. 8 is a view illustrating an example of a structure of thecontent-related information according to an embodiment of the presentdisclosure.

FIG. 9 is a flowchart illustrating an example of a method for changingan arrangement of the content-related information through the displaydevice according to an embodiment of the present disclosure.

FIG. 10 is a flowchart illustrating an example of a method for arrangingother information other than information corresponding to a requestedcategory according to an embodiment of the present disclosure.

FIG. 11 is a view for explaining a first example of arranging thecontent-related information through the display device according to anembodiment of the present disclosure.

FIG. 12 is a view for explaining a second example of arranging thecontent-related information through the display device according to anembodiment of the present disclosure.

FIG. 13 is a view for explaining a third example of arranging thecontent-related information through the display device according to anembodiment of the present disclosure.

FIG. 14 is a view illustrating a first example of content-relatedinformation displayed on a screen according to a voice command throughthe display device according to an embodiment of the present disclosure.

FIG. 15 is a view illustrating a second example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

FIG. 16 is a view illustrating a third example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

FIG. 17 is a view illustrating a fourth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

FIG. 18 is a view illustrating a fifth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

FIGS. 19 and 20 are views illustrating a sixth example ofcontent-related information displayed on the screen according to thevoice command through the display device according to an embodiment ofthe present disclosure.

FIG. 21 is a view illustrating a seventh example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

FIG. 22 is a view illustrating an eighth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

FIG. 23 is a view illustrating a ninth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments relating to the present disclosure will bedescribed in detail with reference to the accompanying drawings. Thesuffixes “module” and “interface” for components used in the descriptionbelow are assigned or mixed in consideration of easiness in writing thespecification and do not have distinctive meanings or roles bythemselves.

FIG. 1 is a block diagram illustrating a configuration of a displaydevice according to an embodiment of the present disclosure.

Referring to FIG. 1 , a display device 100 can include a broadcastreception module 130, an external device interface 135, a storage 140, auser input interface 150, a controller 170, a wireless communicationinterface 173, a voice acquisition module 175, a display 180, an audiooutput interface 185, and a power supply 190.

The broadcast reception module 130 can include a tuner 131, ademodulator 132, and a network interface 133.

The tuner 131 can select a specific broadcast channel according to achannel selection command. The tuner 131 can receive broadcast signalsfor the selected specific broadcast channel.

The demodulator 132 can divide the received broadcast signals into videosignals, audio signals, and broadcast program related data signals andrestore the divided video signals, audio signals, and data signals to anoutput available form.

The network interface 133 can provide an interface for connecting thedisplay device 100 to a wired/wireless network including internetnetwork. The network interface 133 can transmit or receive data to orfrom another user or another electronic device through an accessednetwork or another network linked to the accessed network.

The network interface 133 can access a predetermined webpage through anaccessed network or another network linked to the accessed network. Thatis, it can transmit or receive data to or from a corresponding server byaccessing a predetermined webpage through network.

Then, the network interface 133 can receive contents or data providedfrom a content provider or a network operator. That is, the networkinterface 133 can receive contents such as movies, advertisements,games, VODs, and broadcast signals, which are provided from a contentprovider or a network provider, through network and information relatingthereto.

Additionally, the network interface 133 can receive firmware updateinformation and update files provided from a network operator andtransmit data to an internet or content provider or a network operator.

The network interface 133 can select and receive a desired applicationamong applications open to the air, through network.

The external device interface 135 can receive an application or anapplication list in an adjacent external device and deliver it to thecontroller 170 or the storage 140.

The external device interface 135 can provide a connection path betweenthe display device 100 and an external device. The external deviceinterface 135 can receive at least one of image and audio outputted froman external device that is wirelessly or wiredly connected to thedisplay device 100 and deliver it to the controller. The external deviceinterface 135 can include a plurality of external input terminals. Theplurality of external input terminals can include an RGB terminal, atleast one High Definition Multimedia Interface (HDMI) terminal, and acomponent terminal.

An image signal of an external device inputted through the externaldevice interface 135 can be outputted through the display 180. A soundsignal of an external device inputted through the external deviceinterface 135 can be outputted through the audio output interface 185.

An external device connectable to the external device interface 135 canbe one of a set-top box, a Blu-ray player, a DVD player, a game console,a sound bar, a smartphone, a PC, a USB Memory, and a home theater systembut this is just exemplary.

Additionally, some content data stored in the display device 100 can betransmitted to a user or an electronic device, which is selected fromother users or other electronic devices pre-registered in the displaydevice 100.

The storage 140 can store signal-processed image, voice, or data signalsstored by a program in order for each signal processing and control inthe controller 170.

Additionally, the storage 140 can perform a function for temporarilystore image, voice, or data signals outputted from the external deviceinterface 135 or the network interface 133 and can store information ona predetermined image through a channel memory function.

The storage 140 can store an application or an application list inputtedfrom the external device interface 135 or the network interface 133.

The display device 100 can play content files (for example, video files,still image files, music files, document files, application files, andso on) stored in the storage 140 and provide them to a user.

The user input interface 150 can deliver signals inputted from a user tothe controller 170 or deliver signals from the controller 170 to a user.For example, the user input interface 150 can receive or process controlsignals such as power on/off, channel selection, and screen setting fromthe remote control device 200 or transmit control signals from thecontroller 170 to the remote control device 200 according to variouscommunication methods such as Bluetooth, Ultra Wideband (WB), ZigBee,Radio Frequency (RF), and IR.

Additionally, the user input interface 150 can deliver, to thecontroller 170, control signals inputted from local keys (not shown)such as a power key, a channel key, a volume key, and a setting key.

Image signals that are image-processed in the controller 170 can beinputted to the display 180 and displayed as an image corresponding tocorresponding image signals. Additionally, image signals that areimage-processed in the controller 170 can be inputted to an externaloutput device through the external device interface 135.

Voice signals processed in the controller 170 can be outputted to theaudio output interface 185. Additionally, voice signals processed in thecontroller 170 can be inputted to an external output device through theexternal device interface 135.

Besides that, the controller 170 can control overall operations in thedisplay device 100.

Additionally, the controller 170 can control the display device 100 by auser command or internal program inputted through the user inputinterface 150 and download a desired application or application listinto the display device 100 in access to network.

The controller 170 can output channel information selected by a usertogether with processed image or voice signals through the display 180or the audio output interface 185.

Additionally, according to an external device image playback commandreceived through the user input interface 150, the controller 170 canoutput image signals or voice signals of an external device such as acamera or a camcorder, which are inputted through the external deviceinterface 135, through the display 180 or the audio output interface185.

Moreover, the controller 170 can control the display 180 to displayimages and control broadcast images inputted through the tuner 131,external input images inputted through the external device interface135, images inputted through the network interface, or images stored inthe storage 140 to be displayed on the display 180. In this case, animage displayed on the display 180 can be a still image or video andalso can be a 2D image or a 3D image.

Additionally, the controller 170 can play content stored in the displaydevice 100, received broadcast content, and external input contentinputted from the outside, and the content can be in various formatssuch as broadcast images, external input images, audio files, stillimages, accessed web screens, and document files.

Moreover, the wireless communication interface 173 can perform a wiredor wireless communication with an external electronic device. Thewireless communication interface 173 can perform short-rangecommunication with an external device. For this, the wirelesscommunication interface 173 can support short-range communication byusing at least one of Bluetooth™, Radio Frequency Identification (RFID),Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NearField Communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, andWireless Universal Serial Bus (USB) technologies. The wirelesscommunication interface 173 can support wireless communication betweenthe display device 100 and a wireless communication system, between thedisplay device 100 and another display device 100, or between networksincluding the display device 100 and another display device 100 (or anexternal server) through wireless area networks. The wireless areanetworks can be wireless personal area networks.

Herein, the other display device 100 can be a mobile terminal such as awearable device (for example, a smart watch, a smart glass, and a headmounted display (HMD)) or a smartphone, which is capable of exchangingdata (or inter-working) with the display device 100. The wirelesscommunication interface 173 can detect (or recognize) a communicablewearable device around the display device 100. Furthermore, if thedetected wearable device is a device authenticated to communicate withthe display device 100, the controller 170 can transmit at least part ofdata processed in the display device 100 to the wearable device throughthe wireless communication interface 173. Accordingly, a user of thewearable device can use the data processed in the display device 100through the wearable device.

The voice acquisition module 175 can acquire audio. The voiceacquisition module 175 may include at least one microphone (not shown),and can acquire audio around the display device 100 through themicrophone (not shown).

The display 180 can convert image signals, data signals, or OSD signals,which are processed in the controller 170, or images signals or datasignals, which are received in the external device interface 135, intoR, G, and B signals to generate driving signals.

Furthermore, the display device 100 shown in FIG. 1 is just oneembodiment of the present disclosure and thus, some of the componentsshown can be integrated, added, or omitted according to thespecification of the actually implemented display device 100.

That is, if necessary, two or more components can be integrated into onecomponent or one component can be divided into two or more componentsand configured. Additionally, a function performed by each block is todescribe an embodiment of the present disclosure and its specificoperation or device does not limit the scope of the present disclosure.

According to another embodiment of the present disclosure, unlike FIG. 1, the display device 100 can receive images through the networkinterface 133 or the external device interface 135 and play them withoutincluding the tuner 131 and the demodulator 132.

For example, the display device 100 can be divided into an imageprocessing device such as a set-top box for receiving broadcast signalsor contents according to various network services and a content playbackdevice for playing contents inputted from the image processing device.

In this case, an operating method of a display device according to anembodiment of the present disclosure described below can be performed byone of the display device described with reference to FIG. 1 , an imageprocessing device such as the separated set-top box, and a contentplayback device including the display 180 and the audio output interface185.

The audio output interface 185 receives the audio processed signal fromthe controller 170 and outputs the sound.

The power supply 190 supplies the corresponding power throughout thedisplay device 100. In particular, the power supply 190 supplies powerto the controller 170 that can be implemented in the form of a System OnChip (SOC), a display 180 for displaying an image, and the audio outputinterface 185 for outputting audio or the like.

Specifically, the power supply 190 may include a converter forconverting an AC power source into a DC power source, and a DC/DCconverter for converting a level of the DC source power.

Then, referring to FIGS. 2 and 3 , a remote control device is describedaccording to an embodiment of the present disclosure.

FIG. 2 is a block diagram illustrating a remote control device accordingto an embodiment of the present disclosure and FIG. 3 is a viewillustrating an actual configuration of a remote control deviceaccording to an embodiment of the present disclosure.

First, referring to FIG. 2 , a remote control device 200 can include afingerprint recognition module 210, a wireless communication interface220, a user input interface 230, a sensor 240, an output interface 250,a power supply 260, a storage 270, a controller 280, and a voiceacquisition module 290.

Referring to FIG. 2 , the wireless communication interface 220transmits/receives signals to/from an arbitrary any one of displaydevices according to the above-mentioned embodiments of the presentdisclosure.

The remote control device 200 can include an RF module 221 fortransmitting/receiving signals to/from the display device 100 accordingto the RF communication standards and an IR module 223 fortransmitting/receiving signals to/from the display device 100 accordingto the IR communication standards. Additionally, the remote controldevice 200 can include a Bluetooth module 225 for transmitting/receivingsignals to/from the display device 100 according to the Bluetoothcommunication standards. Additionally, the remote control device 200 caninclude an NFC module 227 for transmitting/receiving signals to/from thedisplay device 100 according to the Near Field Communication (NFC)communication standards and a WLAN module 229 for transmitting/receivingsignals to/from the display device 100 according to the Wireless LAN(WLAN) communication standards

Additionally, the remote control device 200 can transmit signalscontaining information on a movement of the remote control device 200 tothe display device 100 through the wireless communication interface 220.

Moreover, the remote control device 200 can receive signals transmittedfrom the display device 100 through the RF module 221 and if necessary,can transmit a command on power on/off, channel change, and volumechange to the display device 100 through the IR module 223.

The user input interface 230 can be configured with a keypad button, atouch pad, or a touch screen. A user can manipulate the user inputinterface 230 to input a command relating to the display device 100 tothe remote control device 200. If the user input interface 230 includesa hard key button, a user can input a command relating to the displaydevice 100 to the remote control device 200 through the push operationof the hard key button. This will be described with reference to FIG. 3.

Referring to FIG. 3 , the remote control device 200 can include aplurality of buttons. The plurality of buttons can include a fingerprintrecognition button 212, a power button 231, a home button 232, a livebutton 233, an external input button 234, a voice adjustment button 235,a voice recognition button 236, a channel change button 237, a checkbutton 238, and a back button 239.

The fingerprint recognition button 212 can be a button for recognizing auser's fingerprint. According to an embodiment of the presentdisclosure, the fingerprint recognition button 212 can perform a pushoperation and receive a push operation and a fingerprint recognitionoperation. The power button 231 can be button for turning on/off thepower of the display device 100. The power button 231 can be button formoving to the home screen of the display device 100. The live button 233can be a button for displaying live broadcast programs. The externalinput button 234 can be button for receiving an external input connectedto the display device 100. The voice adjustment button 235 can be buttonfor adjusting the size of a volume outputted from the display device100. The voice recognition button 236 can be a button for receivinguser's voice and recognizing the received voice. The channel changebutton 237 can be a button for receiving broadcast signals of a specificbroadcast channel. The check button 238 can be a button for selecting aspecific function and the back button 239 can be a button for returningto a previous screen.

Again, FIG. 2 is described.

If the user input interface 230 includes a touch screen, a user cantouch a soft key of the touch screen to input a command relating to thedisplay device 100 to the remote control device 200. Additionally, theuser input interface 230 can include various kinds of input meansmanipulated by a user, for example, a scroll key and a jog key, and thisembodiment does not limit the scope of the present disclosure.

The sensor 240 can include a gyro sensor 241 or an acceleration sensor243 and the gyro sensor 241 can sense information on a movement of theremote control device 200.

For example, the gyro sensor 241 can sense information on an operationof the remote control device 200 on the basis of x, y, and z axes andthe acceleration sensor 243 can sense information on a movement speed ofthe remote control device 200. Moreover, the remote control device 200can further include a distance measurement sensor and sense a distancewith respect to the display 180 of the display device 100.

The output interface 250 can output image or voice signals correspondingto a manipulation of the user input interface 230 or corresponding tosignals transmitted from the display device 100. A user can recognizewhether the user input interface 230 is manipulated or the displaydevice 100 is controlled through the output interface 250.

For example, the output interface 250 can include an LED module 251 forflashing, a vibration module 253 for generating vibration, a soundoutput module 255 for outputting sound, or a display module 257 foroutputting an image, if the user input interface 230 is manipulated orsignals are transmitted/received to/from the display device 100 throughthe wireless communication interface 220.

Additionally, the power supply 260 supplies power to the remote controldevice 200 and if the remote control device 200 does not move for apredetermined time, stops the power supply, so that power waste can bereduced. The power supply 260 can resume the power supply if apredetermined key provided at the remote control device 200 ismanipulated.

The storage 270 can store various kinds of programs and application datanecessary for a control or operation of the remote control device 200.If the remote control device 200 transmits/receives signals wirelesslythrough the display device 100 and the RF module 221, the remote controldevice 200 and the display device 100 transmits/receives signals througha predetermined frequency band.

The controller 280 of the remote control device 200 can store, in thestorage 270, information on a frequency band for transmitting/receivingsignals to/from the display device 100 paired with the remote controldevice 200 and refer to it.

The controller 280 controls general matters relating to a control of theremote control device 200. The controller 280 can transmit a signalcorresponding to a predetermined key manipulation of the user inputinterface 230 or a signal corresponding to a movement of the remotecontrol device 200 sensed by the sensor 240 to the display device 100through the wireless communication interface 220.

Additionally, the voice acquisition module 290 of the remote controldevice 200 can obtain voice.

The voice acquisition module 290 can include at least one microphone 291and obtain voice through the microphone 291.

Then, FIG. 4 is described.

FIG. 4 is a view of utilizing a remote control device according to anembodiment of the present disclosure.

FIG. 4A illustrates that a pointer 205 corresponding to the remotecontrol device 200 is displayed on the display 180.

A user can move or rotate the remote control device 200 vertically orhorizontally. The pointer 205 displayed on the display 180 of thedisplay device 100 corresponds to a movement of the remote controldevice 200. Since the corresponding pointer 205 is moved and displayedaccording to a movement on a 3D space as show in the drawing, the remotecontrol device 200 can be referred to as a spatial remote controller.

FIG. 4B illustrates that if a user moves the remote control device 200,the pointer 205 displayed on the display 180 of the display device 100is moved to the left in correspondence thereto.

Information on a movement of the remote control device 200 detectedthrough a sensor of the remote control device 200 is transmitted to thedisplay device 100. The display device 100 can calculate the coordinatesof the pointer 205 from the information on the movement of the remotecontrol device 200. The display device 100 can display the pointer 205to match the calculated coordinates.

FIG. 4C illustrates that while a specific button in the remote controldevice 200 is pressed, a user moves the remote control device 200 awayfrom the display 180. Thus, a selection area in the display 180corresponding to the pointer 205 can be zoomed in and displayed largely.

On the other hand, if a user moves the remote control device 200 closeto the display 180, a selection area in the display 180 corresponding tothe pointer 205 can be zoomed out and displayed reduced.

On the other hand, if the remote control device 200 is away from thedisplay 180, a selection area can be zoomed out and if the remotecontrol device 200 is close to the display 180, a selection area can bezoomed in.

Additionally, if a specific button in the remote control device 200 ispressed, the recognition of a vertical or horizontal movement can beexcluded. That is, if the remote control device 200 is moved away fromor close to the display 180, the up, down, left, or right movement maynot be recognized and only the back and fourth movement can berecognized. While a specific button in the remote control device 200 isnot pressed, only the pointer 205 is moved according to the up, down,left or right movement of the remote control device 200.

Moreover, the moving speed or moving direction of the pointer 205 cancorrespond to the moving speed or moving direction of the remote controldevice 200.

Furthermore, a pointer in this specification means an object displayedon the display 180 in correspondence to an operation of the remotecontrol device 200. Accordingly, besides an arrow form displayed as thepointer 205 in the drawing, various forms of objects are possible. Forexample, the above concept includes a point, a cursor, a prompt, and athick outline. Then, the pointer 205 can be displayed in correspondenceto one point of a horizontal axis and a vertical axis on the display 180and also can be displayed in correspondence to a plurality of pointssuch as a line and a surface.

The display device 100 may provide information related to a contentbeing displayed. For example, the display device 100 may provideinformation related to a content by displaying the information relatedto the content being displayed on the display 180 or outputting theinformation through the audio output interface 185. However, a methodfor providing information related to content is not limited to theabove-described method and may be more diverse.

The display device 100 may provide information related to a content thatis being currently viewed by the user.

Hereinafter, for convenience of description, the information related tothe content is referred to as content-related information.

The content-related information may refer to information about products,characters, locations, or music appearing in the content. Thecontent-related information may include information about products,characters, locations, or music similar to those appearing in thecontent, as well as the information about products, characters,locations, or music appearing in the content.

When providing the content-related information, the display device 100may provide content-related information related to a current scene amongthe content-related information. That is, the display device 100 mayprovide information on products, characters, locations, or music relatedto a scene that is currently viewed by the user. As a result, thedisplay device 100 may immediately provide information that the user iscurious about while watching the content.

The user wants to know information about music appearing in the currentscene, but information other than music such as products, characters, orlocations may be provided first in the display device 100. In this case,the user may have to wait until the desired information is provided ormay cause inconvenience in having to separately input the desiredinformation to search the desired information.

Thus, the display device 100 according to the present disclosurepreferentially provides information desired by the user when providingthe content-related information.

FIG. 5 is a flowchart illustrating a method for operating a displaydevice according to an embodiment of the present disclosure.

A controller 170 may display a content (S10).

The controller 170 may display the content on a display 180. In thiscase, the content may be a broadcast image received through a broadcastreception module 130.

However, this is merely an example, and the content may include an imagereceived from an external device such as a set-top box or a smartphone,an image reproduced through an application, and the like.

The controller 170 may determine whether a voice command has beenreceived when displaying the content (S20).

In this case, the voice command may be a command for requestingcontent-related information. That is, the controller 170 may determinewhether the command for requesting the content-related information isreceived when displaying the content.

The user may request the content-related information from the displaydevice 100 by voice through a voice acquisition module 175 provided inthe display device 100 or a voice acquisition module 290 provided in theremote control device 200. Thus, the controller 170 may receive acommand for requesting the content-related information through the voiceacquisition module 175 or the user input interface 150.

When the controller 170 does not receive the voice command, thecontroller 170 may continuously display the content.

When receiving the voice command, the controller 170 may acquire acategory requested by the user based on the voice command (S30).

The requested category may mean a category of the content-relatedinformation requested by the user through the voice command. Forexample, the category may include characters, fashion products, otherproducts, locations, music, and the like, but this is merely an example,and the types of category may vary. The category may be divided intodetailed categories. For example, the character category may havedetailed categories such as men, women, adults, children, etc., and thefashion product category may have detailed categories such as clothes,shoes, pants, etc., but this is merely an example and is not limitedthereto.

According to an embodiment, the controller 170 may acquire the categoryrequested by the user by internally analyzing the voice command. Forexample, the controller 170 may acquire the category requested by theuser by extracting a preset keyword from the voice command. However,this is merely an example and is not limited thereto.

According to another embodiment, the controller 170 may acquire thecategory requested by the user through an external server (e.g., an NLPserver 10 of FIG. 6 ). This will be described with reference to FIG. 6 .

Then, the controller 170 may acquire content-related informationcorresponding to a scene at a time point at which the voice command isreceived (S40).

The controller 170 may acquire content-related information correspondingto a scene at the time point, at which the voice command is received,among the content-related information. In this case, the scene at thetime point, at which the voice command is received, may include only thescene being displayed at the time point, at which the voice command isreceived, or the scene being displayed at the time point, at which thevoice command is received, and scenes for a predetermined time beforereceiving the voice command. Thus, the controller 170 may acquire thecontent-related information corresponding to the scene at the timepoint, at which the voice command is received. Alternatively, thecontroller 170 may acquire content-related information about the sceneat the time point, at which the voice command is received, and thescenes for a predetermined time before receiving the voice command.

The content-related information may be metadata including a category.When receiving the voice command for requesting the content-relatedinformation, the controller 170 may acquire content-related informationthat is metadata including a category.

The controller 170 may acquire content-related information through anexternal server (e.g., the metadata server 40 of FIG. 6 ). This will bedescribed with reference to FIG. 6 .

An order of processes S30 and S40 may be changed. That is, whenreceiving the voice command, the controller 170 may acquire thecontent-related information corresponding to the scene at the timepoint, at which the voice command is received, and then acquire thecategory requested by the user. Also, when receiving the voice command,the controller 170 may simultaneously acquire the content-relatedinformation corresponding to the scene at the time point, at which thevoice command is received, and the category requested by the user.

When the controller 170 acquires the requested category and thecontent-related information according to the voice command, anarrangement of the content-related information may be changed accordingto the requested category (S50), and the content-related information maybe displayed according to the changed arrangement (S60).

That is, the controller 170 may display the content-related informationon a display 180 after changing the arrangement of the content-relatedinformation based on the requested category when displaying thecontent-related information. As described above, when the display device100 changes the arrangement of content-related information based on thecategory requested by the user and displays the content-relatedinformation, since it is highly likely that the information desired bythe user is displayed preferentially, the information desired by theuser may be more quickly and accurately provided.

When the requested category is not acquired, the controller 170 mayarrange the content-related information according to a default order todisplay the content-related information on the display 180.

Next, with reference to FIGS. 6 to 23 , a method for operating thedisplay device 100 described with reference to FIG. 5 and an examplethereof will be described in detail.

FIG. 6 is a schematic view illustrating an example of a method foracquiring a required category and content-related information through adisplay device according to an embodiment of the present disclosure.

The NLP server 10 may be a natural language processing server. The NLPserver 10 may analyze voice data to determine an user's intention. Forexample, the NLP server 10 may analyze the voice data to determine acategory requested by the user.

The metadata server 40 may store and manage content-related informationfor each content. The metadata server 40 may provide the content relatedinformation to the display device 100. For example, when the displaydevice 100 transmits the content information to the metadata server 40,the metadata server 40 may extract the content-related informationaccording to the content information to transmit the content-relatedinformation to the display device 100.

Thus, when the display device 100 receives a user's voice commandrequesting the content-related information, the voice data including thevoice command may be transmitted to the NLP server 10 and receive thecategory requested in the voice command to the NLP server 10.

When the display device 100 receives a user's voice command requestingthe content-related information, the display device 100 may acquirecontent information at a time point at which the voice command isreceived.

In this case, the content information may include at least one of acontent ID for identifying a content, a playback time corresponding tothe time point at which the voice command is received, or a contentcapture image at the time point at which the voice command is received.

When the display device 100 transmits the above-described contentinformation to the metadata server 40, the metadata server 40 mayacquire content-related information based on the content information totransmit the content-related information to the display device 100. Thedisplay device 100 may receive the content related information inresponse to the content information from the metadata server 40. Thecontent-related information may be metadata.

The display device 100 may compare the time point at which the contentinformation (e.g., the captured image) is acquired according to thevoice command and the time point at which the requested categoryappears. When the content-related information at the time point at whichthe content information (e.g., the captured image) is acquired accordingto the voice command includes the information corresponding to therequested category, the display device 100 may change the alignment ofthe content-related information according to the requested category.

FIG. 7 is a view illustrating an example of a method for displayingcontent-related information for each category through the display deviceaccording to an embodiment of the present disclosure.

(a) of FIG. 7 is an example of a form in which content-relatedinformation corresponding to a location category is displayed, and (b)of FIG. 7 is an example of a form in which content-related informationcorresponding to a product category including a character is displayed,and (c) of FIG. 7 is an example of a form in which content-relatedinformation corresponding to a product category, which does not includethe character, is displayed. That is, the product category may includeor may not include a character. Hereinafter, for convenience ofexplanation, it is assumed that ‘product and character categories’ meansa product category including the character, and ‘product category’ meansa product category that does not include the character.

As illustrated in (a) of FIG. 7 , when the category is a location, thecontroller 170 may control the display 180 to display content-relatedinformation including at least one of a location image, a location name,and a location description, and as illustrated in (b) of FIG. 7 , whenthe category is a product including a character, the controller 170 maycontrol the display 180 so that content-related information including atleast one of a character image, a character name, an item worn by thecharacter, or an item similar to the item worn by the character isdisplayed. As illustrated in (c) of FIG. 7 , when the category is aproduct, the controller 170 may control the display 180 so thatcontent-related information including at least one of an item includedin the scene or an item similar to the item included in the scene isdisplayed.

Content-related information may be generated for each character asinformation corresponding to product and character categories.

The controller 170 may control the display 180 to preferentially displaycontent-related information corresponding to a requested category whendisplaying the content-related information. For example, when therequested category is the location, the controller 170 may control thedisplay 180 so that the content-related information illustrated in (a)of FIG. 7 is preferentially displayed rather than the content-relatedinformation illustrated in (b) of FIG. 7 or the content-relatedinformation illustrated in (c) of FIG. 7 .

FIG. 8 is a view illustrating an example of a structure of thecontent-related information according to an embodiment of the presentdisclosure.

FIG. 8 is a view illustrating an example of content-related informationthat is metadata, and the content-related information may includecontent information and a category.

The content information (media) may include a media id, a media name,etc. corresponding to each frame time for each of a plurality of framesconstituting the content. The media id may be a content ID, and themedia name may be a content name.

The category may include a location, a product, and others. In addition,the category may further include a character. That is, the category mayinclude at least one of a location, a product, a character, or others.

According to an example, the character category may exist separatelyfrom the location category, product category, and other categories.According to another example, the character category may be included inthe product category. In this case, the product category may include ormay not include the character category.

The location may include a location name, a location image, and alocation link corresponding to each frame time for each of a pluralityof frames constituting the content.

The product may include character/actor names, a character gender, acharacter image, and a character rank corresponding to each frame timefor each of the plurality of frames constituting the content. Inaddition, each of the characters/actor names may include a product rank,a product category, a product name, a product image, a product price, aproduct brand, a product link, a similar product, and the similarproduct may include a similar product name, a similar product image, asimilar product price, a similar product brand, and a similar productlink.

For example, the product category may include a ‘clothing’ category, a‘shoes’ category, and a ‘jewelry’ category.

Others may include a product rank, a product category, a product name, aproduct image, a product price, a product brand, a product link, and asimilar product corresponding to each frame time for each of theplurality of frames constituting the content, and the similar productmay include a similar product name, a similar product image, a similarproduct price, a similar product brand, and a similar product link.

That is, the content-related information may include categoriescorresponding to the frame time for each of the plurality of framesconstituting the content. That is, the content-related information thatis metadata may be categorized.

The controller 170 may more easily acquire content-related informationcorresponding to a scene at a time point, at which the voice command isreceived, by using the content-related information having the structureillustrated in FIG. 8 . For example, the controller 170 may acquire aplayback time of the content corresponding to the time point, at whichthe voice command is received, and acquire information about a location,a product, and others, which correspond to the frame time that matchesthe playback time of the content.

Then, the controller 170 may acquire information corresponding to therequested category by using the content-related information having astructure as shown in FIG. 8 . For example, the controller 170 mayacquire information corresponding to the requested category byextracting information that matches the information corresponding to therequested category from the content-related information having thestructure illustrated in FIG. 8 .

When the information corresponding to the requested category isacquired, the controller 170 may display the content-related informationafter changing the arrangement of the content-related information sothat the information corresponding to the requested category isdisplayed preferentially rather than other information.

Next, an example of a method for changing the arrangement of thecontent-related information by the display device 100 according to anembodiment of the present disclosure will be described with reference toFIG. 9 .

FIG. 9 is a flowchart illustrating an example of a method for changingan arrangement of the content-related information through the displaydevice according to an embodiment of the present disclosure.

Particularly, FIG. 9 may be a flowchart in which an operation S50 ofFIG. 5 is embodied.

The controller 170 may determine whether information corresponding tothe requested category exists in the content-related information (S51).

Specifically, the controller 170 may detect a frame time correspondingto the content playback time at a time point, at which the voice commandis received, and search information corresponding to the requestedcategory among the categories corresponding to the frame time detectedin the content-related information. If the information corresponding tothe requested category is not searched from the content-relatedinformation, the controller 170 may determine that the informationcorresponding to the requested category does not exist in thecontent-related information, and if the information corresponding to therequested category is searched from the content-related information, thecontroller 170 may determine that the information corresponding to therequested category exists in the content-related information.

If the information corresponding to the requested category exists in thecontent-related information, the controller 170 may arrange theinformation corresponding to the requested category in priority orderrather than other information (S53).

In this case, the other information may be information that does notcorrespond to the requested category.

The controller 170 may arrange information corresponding to therequested category in priority order rather than the information thatdoes not correspond to the requested category.

According to an embodiment, the controller 170 may arrange thecontent-related information so that the information corresponding to therequested category is located in priority order rather than theinformation that does not correspond to the requested category. Forexample, when the content-related information includes informationcorresponding to a first category and information corresponding to asecond category, if the requested category is the first category, thedisplay 180 may display the information corresponding to the firstcategory in priority order rather than the information corresponding tothe second category.

According to another embodiment, the controller 170 may arrange thecontent-related information so that a display size of the informationcorresponding to the requested category is larger than a display size ofthe information that does not correspond to the requested category. Forexample, when the content-related information includes informationcorresponding to the first category and information corresponding to thesecond category, if the requested category is the first category, thedisplay 180 may display the information corresponding to the firstcategory having a size greater than that of the informationcorresponding to the second category.

The above-described method is merely exemplary, and the controller 170may arrange the information corresponding to the requested category inpriority order rather than other information in various manners.

If the information corresponding to the requested category does notexist in the content-related information, the controller 170 may arrangeother information in addition to the information corresponding to therequested category (S55).

If the information corresponding to the requested category does notexist in the content-related information, the controller 170 may arrangethe content-related information so that other information is displayed.

In this case, the other information may include at least one of thatinformation that does not correspond to the requested category or theinformation corresponding to the requested category existing in thescene before the time point at which the voice command is received.

Next, an example of a method for arranging information other than thatinformation corresponding to the requested category by the displaydevice 100 according to an embodiment of the present disclosure will bedescribed with reference to FIG. 10 .

FIG. 10 is a flowchart illustrating an example of a method for arrangingother information other than information corresponding to a requestedcategory according to an embodiment of the present disclosure.

Particularly, FIG. 10 is a flowchart in which an operation S55 of FIG. 9is embodied.

The controller 170 may determine whether other information in additionto the information corresponding to the requested category exists (S61).

If the information corresponding to the requested category does notexist in the content-related information, the controller 170 maydetermine whether other information exists in addition to theinformation corresponding to the requested category.

If other information exists in addition to the information correspondingto the requested category, the controller 170 may arrange otherinformation in addition to the information corresponding to therequested category (S63).

In this case, the other information may be information corresponding toa category other than the requested category as that information thatdoes not correspond to the requested category. That is, when thatinformation that does not correspond to the requested category exists inaddition to that information corresponding to the requested category,the controller 170 may arrange the information that does not correspondto the requested category. That is, when information corresponding to acategory other than the requested category exists in the content-relatedinformation, the controller 170 may control the display 180 to displayinformation corresponding to the other category.

If there is no other information other than information corresponding tothe requested category, the controller 170 may determine whether thatinformation corresponding to the requested category exists in theprevious scene (S65).

The previous scene may refer to scenes for a predetermined time beforereceiving the voice command. For example, the previous scene may referto scenes for 5 seconds before receiving the voice command. However, inthis case, the time of 5 seconds is merely an example, and thus, it isappropriate that the present disclosure is not limited thereto.

Specifically, the controller 170 may detect a frame time correspondingto each of the previous scenes and search information corresponding to arequested category among categories corresponding to the frame times.

Regardless of operation S61, if the information corresponding to therequested category does not exist in the content-related information, itmay be determined whether information corresponding to the requestedcategory exists in the scene before receiving the voice command.

If information corresponding to the requested category exists in theprevious scene, the controller 170 may arrange information correspondingto the requested category included in the previous scene (S67).

That is, if the information corresponding to the requested categoryexists in the scene before receiving the voice command, the controller170 may control the display 180 to display the information correspondingto the requested category included in the previous scene.

However, if there is no information corresponding to the requestedcategory in the previous scene, the controller 170 may arrange allinformation included in the content (S69).

That is, if there is no information corresponding to the requestedcategory in the previous scene, the controller 170 may arrange thecontent-related information so that all of the content-relatedinformation are displayed.

If the scene at the time point, at which the voice command is received,and the information corresponding to the category requested in theprevious scene are not included, the controller 170 may control anentire view mode of the content-related information in which all of thecontent-related information are displayed.

Regardless of operations S61 and S65, if the information correspondingto the requested category does not exist in the content-relatedinformation, the controller 170 may control the display 180 so that allof the content-related information included in the content aredisplayed.

Next, FIG. 11 is a view for explaining a first example of arranging thecontent-related information through the display device according to anembodiment of the present disclosure.

In the example of FIG. 11 , the frame time corresponding to the contentplayback time at the time point, at which the voice command is received,may be 10,000 ms, and the requested category may be ‘shoes’. Thecontroller 170 may search information corresponding to the requestedcategory ‘shoes’ among categories corresponding to the frame time of10000 ms.

According to the example illustrated in (a) of FIG. 11 , thecontent-related information may include information on ‘Product 1’ of‘Character 1’ corresponding to the category ‘shoes’ corresponding to theframe time of 10000 ms. Thus, in this case, the controller 170 mayarrange the information on ‘Product 1’ of ‘Character 1’ corresponding tothe category ‘shoes’ corresponding to the frame time of 10,000 ms inpriority order rather than other information, i.e., information about‘Product 2’ of ‘Character 1’, information on ‘Product 1’ of ‘Character2’, and information on ‘location’.

According to the example illustrated in (b) of FIG. 11 , thecontent-related information may not include information corresponding tothe category ‘shoes’ corresponding to the frame time of 10,000 ms. Thus,in this case, the controller 170 may display information on ‘Product 1’of ‘Character 1’ corresponding to another category ‘clothing’corresponding to frame time 10,000 ms first and display information on‘Product 2’ of the category ‘Character 1’ corresponding to a frame time9,000 ms, which is the previous scene and then display information on‘Product 1’ of ‘Character 2’ and information on ‘location’.

As described above, the controller 170 may arrange the content-relatedinformation so that information preferred by the user is located in thepriority order based on the requested category and playback time.

Next, FIG. 12 is a view for explaining a second example of arranging thecontent-related information through the display device according to anembodiment of the present disclosure.

The content-related information may be set to be arranged in an order ofa location category, product and character categories, and a productcategory by defaults.

Thus, if there is no requested category, as illustrated in (a) of FIG.12 , the controller 170 may arrange the content-related information soas to be displayed in order of information corresponding to the locationcategory, information corresponding to the product and charactercategories, and information corresponding to the product category.

When the requested category is a character, as illustrated (b) of FIG.12 , the controller 170 may arrange the content-related information sothat the information corresponding to the product and person categoriesare displayed in priority order rather than the informationcorresponding to the location category and the information correspondingto the product category.

FIG. 13 is a view for explaining a third example of arranging thecontent-related information through the display device according to anembodiment of the present disclosure.

In the information corresponding to the product and charactercategories, it may be set to be arranged in order of a ‘Clothing’category, a ‘Shoes’ category, and a ‘Jewelry’ category by defaults.

Thus, when the requested category may include at least one of ‘Product’or ‘Fashion’, as illustrated in (a) of FIG. 13 , the controller 170 mayarrange the content-related information so as to be displayed in orderof information corresponding to ‘Clothing’ category, informationcorresponding to a ‘Shoes’ category, and information corresponding to a‘Jewelry’ category in the information corresponding to the product andcharacter categories. That is, the controller 170 may arrange thecontent-related information so that the information corresponding to the‘Clothing’ category is displayed, and then, the informationcorresponding to the ‘Shoes’ category and the information correspondingto the ‘Jewelry’ category are sequentially displayed whenever each timeat which a next command is received, within the informationcorresponding to the product and character categories. In summary, thecontroller 170 may change the order of the content-related informationdisplayed in the information corresponding to the product and charactercategories according to the requested category.

When the voice command is “What is that jewelry?”, the controller 170may acquire ‘Product’, ‘Fashion’, and ‘Jewelry’ as the requestedcategory. In this case, the controller 170 may arrange thecontent-related information so that the information corresponding to the‘Jewelry’ category is displayed in priority order rather than theinformation corresponding to the ‘Clothing’ category and the informationcorresponding to the ‘Shoes’ category within the informationcorresponding to the product and character categories. That is, thecontroller 170 may arrange the content-related information so that theinformation corresponding to the ‘Jewelry’ category is displayed, andthen, the information corresponding to the ‘Clothing’ category and theinformation corresponding to the ‘Shoes’ category are sequentiallydisplayed whenever each time at which a next command is received, withinthe information corresponding to the product and character categories.

Next, a method for displaying the content-related information accordingto the voice command by the display device according to an embodiment ofthe present disclosure will be described with reference to FIGS. 14 to23 through various examples.

FIG. 14 is a view illustrating a first example of the content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

As illustrated in (a) of FIG. 14 , the controller 170 may receive “Whois that characters?” as the voice command. The controller 170 mayacquire a captured image by capturing the screen at the time point, atwhich the voice command is received.

The controller 170 may acquire the requested category based on the voicecommand. For example, the controller 170 may acquire ‘character/actor’as a requested category based on the voice command.

In addition, the controller 170 may acquire content-related informationbased on the captured image. The controller 170 may display thecontent-related information on the display 180. Here, thecontent-related information may be arranged according to the requestedcategory.

An example illustrated in (b) of FIG. 14 may be an example of a screenon which the content-related information is displayed together with acaptured image 1401 through the display 180.

The captured image 1401 may be a video image captured when the voicecommand is received. Through the captured image 1401, there is anadvantage that the user easily checks which scene the content-relatedinformation is related to the content-related information.

Also, in the example of FIG. 14 , the content-related informationcorresponding to the captured image 1401 may include information 1403corresponding to product and character categories and information 1405corresponding to a location category.

In this case, the display 180 may display the information 1403corresponding to the product and character categories in priority orderrather than the information 1405 corresponding to the location categorybased on the requested category.

The information 1403 corresponding to the product and charactercategories may include information about a product worn by a characterappearing in the content and information about a product similar to thatworn by the character.

The information 1403 corresponding to the product and charactercategories may be different for each character. For example, when thecharacter appearing in the content is a first character and a secondcharacters, information corresponding to the product and charactercategories for the first character and information corresponding to theproduct and character categories for the second character may separatelyexist.

The information 1403 corresponding to the product and charactercategories may include a previous view icon 1403 a and a next view icon1403 b. The previous view icon 1403 a and the next view icon 1403 b maybe icons for switching to display a product other than the displayedproduct when the information corresponding to product and charactercategories includes a plurality of products.

The information 1405 corresponding to the location category may includeinformation about a location appearing in the content. Particularly, theinformation 1405 corresponding to the location category may includeinformation about a location appearing in the captured image 1401.

In addition, the display 180 may further display an entire view icon1409. The entire view icon 1409 may be an icon for displaying all of thecontent-related information included in the currently displayed content.

Thus, there is an advantage that the user preferentially checks therequested information.

FIG. 15 is a view illustrating a second example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

As illustrated in (a) of FIG. 15 , the controller 170 may receive “Whereis that location?” as the voice command. The controller 170 may acquirea captured image by capturing the screen at the time point, at which thevoice command is received.

The controller 170 may acquire the requested category based on the voicecommand. For example, the controller 170 may acquire ‘location’ as arequested category based on the voice command.

In addition, the controller 170 may acquire content-related informationbased on the captured image. The controller 170 may display thecontent-related information on the display 180. Here, thecontent-related information may be arranged according to the requestedcategory.

An example illustrated in (b) of FIG. 15 may be an example of a screenon which the content-related information is displayed together with acaptured image 1501 through the display 180.

The captured image 1501 may be a video image captured when the voicecommand is received. Through the captured image 1501, there is anadvantage that the user easily checks which scene the content-relatedinformation is related to the content-related information.

Also, in the example of FIG. 15 , the content-related informationcorresponding to the captured image 1501 may include information 1503corresponding to a location category and information 1505 correspondingto production and character categories.

In this case, the display 180 may display the information 1503corresponding to the location category in priority order rather than theinformation 1505 corresponding to the product and character categories.

Similarly, the information 1503 corresponding to the location categorymay include information about a location appearing in the content.Particularly, the information 1503 corresponding to the locationcategory may include information about a location appearing in thecaptured image 1501.

The information 1505 corresponding to the product and charactercategories may include information about a product worn by a characterappearing in the content and information about a product similar to thatworn by the character.

The information 1505 corresponding to the product and charactercategories may include a previous view icon 1505 a and a next view icon1505 b. The previous view icon 1505 a and the next view icon 1505 b maybe icons for switching information to display a product other than thedisplayed product when the information corresponding to product andcharacter categories includes a plurality of products.

In addition, the display 180 may further display an entire view icon1509. The entire view icon 1509 may be an icon for displaying all of thecontent-related information included in the currently displayed content.

Thus, there is an advantage that the user preferentially checks therequested information.

FIG. 16 is a view illustrating a third example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

As illustrated in (a) of FIG. 16 , the controller 170 may receive “Whois that characters?” as the voice command. The controller 170 mayacquire a captured image by capturing the screen at the time point, atwhich the voice command is received.

The controller 170 may acquire the requested category based on the voicecommand. For example, the controller 170 may acquire ‘character/actor’as a requested category based on the voice command.

In addition, the controller 170 may acquire content-related informationbased on the captured image. The controller 170 may display thecontent-related information on the display 180. Here, thecontent-related information may be arranged according to the requestedcategory.

An example illustrated in (b) of FIG. 16 may be an example of a screenon which the content-related information is displayed together with acaptured image 1601 through the display 180.

The captured image 1601 may be a video image captured when the voicecommand is received. Through the captured image 1601, there is anadvantage that the user easily checks which scene the content-relatedinformation is related to the content-related information.

Also, in the example of FIG. 16 , the content-related informationcorresponding to the captured image 1601 may include information 1603corresponding to product and character categories on the first characterand information 1605 corresponding to product and character categorieson the second character.

In this case, the display 180 may display any one of the information1603 corresponding to the product and character categories for the firstcharacter and the information 1605 corresponding to the product andcharacter categories for the second character in priority order ratherthan the other based on the requested category.

Whether the information corresponding to the product and charactercategories for which one of the first character and the second characteris displayed first may be determined according to a character rank amongthe categories. In the example of FIG. 16 , it is assumed that theinformation 1603 corresponding to the product and character category forthe first character is displayed in priority order rather than theinformation 1605 corresponding to the product and character categoriesfor the second character.

Each of the pieces of information 1603 and 1605 corresponding to theproduct and character categories for the first and second characters mayinclude information on products worn by the character appearing in thecontent and information on products similar to those worn by thecharacter.

The information 1603 corresponding to the product and charactercategories of the first character and the information 1605 correspondingto the product and character categories of the second character may bedifferent from each other.

In the information 1603 corresponding to the product and charactercategories for the first character, there may be a plurality of piecesof information of the ‘product’ category related to the first character.In this case, the controller 170 may arrange the content-relatedinformation so that the information of the ‘Clothing’ category among theplurality of ‘product’ categories related to the first character isdisplayed preferentially. That is, when the requested category includesonly the ‘product’ category, the controller 170 may determine the orderof information belonging to the ‘product’ category according todefaults.

In addition, the information 1603 corresponding to the product andcharacter categories of the first character may include a previous viewicon 1603 a and a next view icon 1603 b. The previous view icon 1603 aand the next view icon 1603 b may be icons for switching information ofthe plurality of ‘product’ categories to be sequentially displayed.

When the controller 170 displays the information 1605 corresponding tothe product and character categories for the second character, insimilar to the information 1603 corresponding to the product andcharacter categories for the first character, the order of the pieces ofinformation belonging to the ‘product’ category may be determined andinclude the previous view icon (not shown) and the next view icon (notshown).

FIG. 17 is a view illustrating a fourth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

As illustrated in (a) of FIG. 17 , the controller 170 may receive “Whatis that jewelry?” as the voice command. The controller 170 may acquire acaptured image by capturing the screen at the time point, at which thevoice command is received.

The controller 170 may acquire the requested category based on the voicecommand. For example, the controller 170 may acquire at least one of‘Product’, ‘Fashion’, or

‘Jewelry’ as the requested category based on the voice command.

In addition, the controller 170 may acquire content-related informationbased on the captured image. The controller 170 may display thecontent-related information on the display 180. Here, thecontent-related information may be arranged according to the requestedcategory.

An example illustrated in (b) of FIG. 17 may be an example of a screenon which the content-related information is displayed together with acaptured image 1701 through the display 180.

The captured image 1701 may be a video image captured when the voicecommand is received. Through the captured image 1701, there is anadvantage that the user easily checks which scene the content-relatedinformation is related to the content-related information.

Also, in the example of FIG. 17 , the content-related informationcorresponding to the captured image 1701 may include information 1703corresponding to product and character categories on the first characterand information 1705 corresponding to product and character categorieson the second character.

In this case, the display 180 may display any one of the information1703 corresponding to the product and character categories for the firstcharacter and the information 1705 corresponding to the product andcharacter categories for the second character in priority order ratherthan the other based on the requested category. Similarly to FIG. 16 ,in the example of FIG. 16 , it is assumed that the information 1703corresponding to the product and character category for the firstcharacter is displayed in priority order rather than the information1705 corresponding to the product and character categories for thesecond character.

Each of the pieces of information 1703 and 1705 corresponding to theproduct and character categories for the first and second characters mayinclude information on products worn by the character appearing in thecontent and information on products similar to those worn by thecharacter.

The information 1703 corresponding to the product and charactercategories of the first character and the information 1705 correspondingto the product and character categories of the second character may bedifferent from each other.

In the information 1703 corresponding to the product and charactercategories for the first character, there may be a plurality of piecesof information of the ‘product’ category related to the first character.In this case, the controller 170 may arrange the content-relatedinformation so that the information of the ‘Jewelry’ category among theplurality of ‘product’ categories related to the first character isdisplayed preferentially. That is, since the requested category includesthe ‘Jewelry’ category, the controller 170 may arrange thecontent-related information so that the information corresponding to the‘Jewelry’ category among information belonging to the ‘product’ categoryis displayed first according to the requested category.

In addition, the information 1703 corresponding to the product andcharacter categories of the first character may include a previous viewicon 1703 a and a next view icon 1703 b. The previous view icon 1703 aand the next view icon 1703 b may be icons for switching information ofthe plurality of ‘product’ categories to be sequentially displayed. Forexample, when the controller 170 receives a command for selecting thenext view icon 1703 b while the information corresponding to the‘Jewelry’ category is displayed, the controller 170 may control thedisplay 180 so that other information belonging to the ‘product’category is displayed.

Similarly, when the controller 170 displays the information 1705corresponding to the product and character categories for the secondcharacter, in similar to the information 1703 corresponding to theproduct and character categories for the first character, the controller170 may arrange the content-related information so that the informationcorresponding to the ‘Jewelry’ category is displayed in priority orderrather than the pieces of other information and include the previousview icon (not shown) and the next view icon (not shown).

FIG. 18 is a view illustrating a fifth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

As illustrated in (a) of FIG. 18 , the controller 170 may receive “Whatis that clothing?” as the voice command. The controller 170 may acquirea captured image by capturing the screen at the time point, at which thevoice command is received.

The controller 170 may acquire the requested category based on the voicecommand. For example, the controller 170 may acquire at least one of‘Product’, ‘Fashion’, or ‘Clothing’ as the requested category based onthe voice command.

In addition, the controller 170 may acquire content-related informationbased on the captured image. The controller 170 may display thecontent-related information on the display 180. Here, thecontent-related information may be arranged according to the requestedcategory.

An example illustrated in (b) of FIG. 18 may be an example of a screenon which the content-related information is displayed together with acaptured image 1801 through the display 180.

The captured image 1801 may be a video image captured when the voicecommand is received. Through the captured image 1801, there is anadvantage that the user easily checks which scene the content-relatedinformation is related to the content-related information.

Also, in the example of FIG. 18 , the content-related informationcorresponding to the captured image 1801 may include information 1803corresponding to product and character categories on the first characterand information 1805 corresponding to a product category. Theinformation 1803 corresponding to the product and character categoriesof the first character may include a previous view icon 1803 a and anext view icon 1803 b.

In this case, the information 1803 corresponding to the product andcharacter categories for the first character may include informationcorresponding to the ‘Fashion’ category, and the information 1805corresponding to the product category may include informationcorresponding to ‘others’ categories. In this case, the display 180 maydisplay the information 1803 corresponding to the product and charactercategories for the first character in priority order rather than theinformation 1805 corresponding to the product category based on therequested category.

However, unlike the example of FIG. 18 , when the controller 170receives “What is that furniture?” as the voice command, at least one of‘others’ or ‘furniture’ may be acquired as the requested category.

In this case, the display 180 may display the information 1805corresponding to the product category in priority order rather than theinformation 1803 corresponding to the product and character categoriesfor the first character based on the requested category.

FIGS. 19 and 20 are views illustrating a sixth example ofcontent-related information displayed on the screen according to thevoice command through the display device according to an embodiment ofthe present disclosure.

As illustrated in (a) of FIG. 19 , the controller 170 may receive “Whatis that men's clothing?” as the voice command. The controller 170 mayacquire a captured image by capturing the screen at the time point, atwhich the voice command is received.

The controller 170 may acquire the requested category based on the voicecommand. For example, the controller 170 may acquire at least one of‘Product’, ‘Fashion’, ‘Clothing’, or ‘men’ as the requested categorybased on the voice command.

In addition, the controller 170 may acquire content-related informationbased on the captured image. The controller 170 may display thecontent-related information on the display 180. Here, thecontent-related information may be arranged according to the requestedcategory.

An example illustrated in (b) of FIG. 19 may be an example of a screenon which the content-related information is displayed together with acaptured image 1901 through the display 180.

The captured image 1901 may be a video image captured when the voicecommand is received. Through the captured image 1901, there is anadvantage that the user easily checks which scene the content-relatedinformation is related to the content-related information.

Also, in the example of FIG. 19 , the content-related informationcorresponding to the captured image 1901 may include information 1903corresponding to product and character categories on the first characterand information 1905 corresponding to product and character categorieson the second character. In this case, the information 1903corresponding to the product and character categories for the firstcharacter may include a ‘men’ category, and the information 1905corresponding to the product and character categories for the secondcharacter may include a ‘woman’ category. In this case, the display 180may display the information 1903 corresponding to the product andcharacter categories for the first character and the information 1905corresponding to the product and character categories for the secondcharacter in priority order based on the requested category.

As illustrated in FIG. 20 , the controller 170 may receive “What is thatwomen's clothing?” as the voice command. For example, the controller 170may acquire at least one of ‘Product’, ‘Fashion’, ‘Clothing’, or ‘women’as the requested category based on the voice command.

In this case, the controller 170 may arrange the content-relatedinformation so that the information 1905 corresponding to the productand character categories for the second character including the ‘women’category is displayed in priority order rather than the information 1903corresponding to the product and character categories for the firstcharacter including the ‘men’ category.

FIG. 21 is a view illustrating a seventh example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

The controller 170 may acquire the requested category according to thevoice command, but the information corresponding to the requestedcategory may not exist in the content-related information. In this case,the display 180 may display a message indicating that there is noinformation corresponding to the requested category. For example, asillustrated in (a) of FIG. 21 , the display 180 may display a messagesuch as “No information for the item you said”.

Then, the controller 170 may control the display 180 to further displaya message that asks whether to confirm the information contained in theprevious scene in addition to the message indicating that there is noinformation corresponding to the requested category, a no icon 2100 a,and a yes icon 2100

For example, the display 180 may display a message such as “Would youlike to take a look at information in the previous scene?”

When receiving a selection command of the no icon 2100 a, the controller170 may not display the content-related information.

However, when receiving the selection command of the yes icon 2100 b,the controller 170 may acquire the information corresponding to thecategory requested in the previous scene. Particularly, the controller170 may extract a scene having the information corresponding to therequested category from among the previous scenes. If there are aplurality of scenes having the information corresponding to therequested category among the previous scenes, the controller 170 mayextract one scene closest to a time point, at which the voice command isreceived, from among the plurality of scenes. The controller 170 maydisplay the content related information included in the previouslyextracted scene as described above.

For example, as illustrated in (b) of FIG. 21 , the controller 170 maydisplay a captured image 2101 of the previous scene, information 2103corresponding to product and character categories included in theprevious scene, and an entire view icon 2105.

In this case, it is possible to minimize a problem in which informationdesired by the user is not displayed as user utters a bit late, andthus, there is an advantage in that information desired by the user ismore accurately provided.

According to an embodiment, when the information corresponding to therequested category does not exist in the content-related information,the controller 170 omits the screen illustrated in (a) of FIG. 21 andimmediately may display a screen as illustrated in (b) of FIG. 21 .

FIG. 22 is a view illustrating an eighth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

The controller 170 may acquire the requested category according to thevoice command, but the information corresponding to the requestedcategory may not exist in the content-related information. In this case,the display 180 may display a message indicating that there is noinformation corresponding to the requested category. For example, asillustrated in (a) of FIG. 22 , the display 180 may display a messagesuch as “No information for the item you said”.

Then, the controller 170 may control the display 180 to further displaya message that asks wither to confirm the information contained in thecurrent scene in addition to the message indicating that there is noinformation corresponding to the requested category, a no icon 2200 a,and a yes icon 2200 b.

For example, the display 180 may display a message such as “Would youlike to take a look at other information in the scene?”

When receiving a selection command of the no icon 2200 a, the controller170 may not display the content-related information.

However, when receiving the selection command of the yes icon 2200 b,the controller 170 may acquire the information that does not correspondto the category requested in the current scene. The current scene maymean a scene being displayed at the time point at which the voicecommand is received. The controller 170 may extract information otherthan the information corresponding to the requested category from amongthe current scene.

As illustrated in (b) of FIG. 22 , the controller 170 may display acaptured image 2201 of the current scene, other information 2203 inaddition to the information corresponding to the requested category, andan entire view icon 2205.

In this case, it has an advantage of providing a product similar to theproduct requested by the user instead of providing any information tothe user by displaying other information related to the scene that isbeing watched by the user even though it does not exactly match thecategory requested by the user.

According to an embodiment, when the information corresponding to therequested category does not exist in the content-related informationcorresponding to the current scene, the controller 170 omits the screenillustrated in (a) of FIG. 22 and immediately may display a screen asillustrated in (b) of FIG. 22 .

FIG. 23 is a view illustrating a ninth example of content-relatedinformation displayed on the screen according to the voice commandthrough the display device according to an embodiment of the presentdisclosure.

The controller 170 may acquire the requested category according to thevoice command, but the information corresponding to the requestedcategory may not exist in the content-related information. In this case,the display 180 may display a message indicating that there is noinformation corresponding to the requested category. For example, asillustrated in (a) of FIG. 23 , the display 180 may display a messagesuch as “No information for the item you said”.

Then, the controller 170 may control the display 180 to further displaya message that asks whether to execute the entire view in addition tothe message indicating that there is no information corresponding to therequested category, a no icon 2300 a, and a yes icon 2300 b.

The entire view may be a mode for displaying all content-relatedinformation included in the currently viewed image.

For example, the display 180 may display a message such as “Would youlike to take a look at all information in the content?”.

When receiving a selection command of the no icon 2300 a, the controller170 may not display the content-related information.

However, when receiving the selection command of the yes icon 2300 b,the controller 170 may control the display 180 to display allcontent-related information included in the currently displayed content.

As illustrated in (b) of FIG. 23 , the display 180 may display an entireview menu 2310, and the entire view menu 2310 may include information2311 corresponding to a location category, information 2313corresponding to product and character categories for the firstcharacter, and information 2315 corresponding to the product andcharacter categories for the second character.

In this case, when the information requested by the user is notsearched, a function for directly searching the information desired bythe user may be provided.

Unlike described with reference to FIGS. 21 to 23 , if the informationcorresponding to the requested category does not exist in thecontent-related information, the controller 170 may operate according tothe flowchart illustrated in FIG. 10 to control the display 180 so thata screen as illustrated in (b) of FIG. 22 is displayed in operation S63,a screen as illustrated in (b) of FIG. 21 is displayed in operation S67,and a screen as illustrated in (b) of FIG. 23 is displayed in operationS69.

The above description is merely illustrative of the technical idea ofthe present disclosure, and various modifications and changes may bemade thereto by those skilled in the art without departing from theessential characteristics of the present disclosure.

Therefore, the embodiments of the present disclosure are not intended tolimit the technical spirit of the present disclosure but to illustratethe technical idea of the present disclosure, and the technical spiritof the present disclosure is not limited by these embodiments.

The scope of protection of the present disclosure should be interpretedby the appending claims, and all technical ideas within the scope ofequivalents should be construed as falling within the scope of thepresent disclosure.

1. A display device comprising: a display configured to display acontent; and a controller configured to control the display to displaycontent-related information corresponding to a scene at a time point, atwhich a voice command is received, when the voice command is received,wherein the controller changes an arrangement of the content-relatedinformation on the basis of a category requested by the voice command.2. The display device according to claim 1, wherein the controller isconfigured to change the arrangement of the content-related informationso that the information corresponding to the requested category isdisplayed in priority order rather than other information.
 3. Thedisplay device according to claim 1, wherein, when the content-relatedinformation comprises information corresponding to a first category andinformation corresponding to a second category, if the requestedcategory is the first category, the display is configured to display theinformation corresponding to the first category in priority order ratherthan the information corresponding to the second category.
 4. Thedisplay device according to claim 1, wherein, when the informationcorresponding to the requested category does not exist in thecontent-related information, the controller is configured to control thedisplay so that other information other than the informationcorresponding to the requested category are displayed.
 5. The displaydevice according to claim 1, wherein, when the information correspondingto the requested category does not exist in the content-relatedinformation, the controller is configured to determine whetherinformation corresponding to other categories other than the requestedcategory exist in the content-related information.
 6. The display deviceaccording to claim 5, wherein, when information corresponding to othercategories other than the requested category exists in thecontent-related information, the controller is configured to control thedisplay so that the information corresponding to the other categories isdisplayed.
 7. The display device according to claim 7, wherein, when theinformation corresponding to the requested category does not exist inthe content-related information, the controller is configured todetermine whether the information corresponding to the requestedcategory exists on a previous scene before receiving the voice command.8. The display device according to claim 7, wherein, when theinformation corresponding to the requested category exists on the scenebefore receiving the voice command, the controller is configured tocontrol the display so that the information corresponding to therequested category comprised in the previous scene is displayed.
 9. Thedisplay device according to claim 1, wherein, when the informationcorresponding to the requested category does not exist in thecontent-related information, the controller is configured to control thedisplay so that all of the content-related information comprised in thecontent is displayed.
 10. The display device according to claim 1,wherein, when the voice command for requesting the content-relatedinformation is received, the controller is configured to acquire thecontent-related information that is metadata including a category. 11.The display device according to claim 1, wherein the category comprisesat least one of a location, a product, a character, and others.
 12. Thedisplay device according to claim 1, wherein the content-relatedinformation comprises categories corresponding to a frame time for eachof a plurality of frames constituting the content.
 13. The displaydevice according to claim 1, wherein, when the content-relatedinformation is displayed, the controller is configured to control thedisplay so that an image captured at a time point, at which the voicecommand is received, is further displayed.
 14. The display deviceaccording to claim 1, wherein the controller is configured to acquirethe requested category based on the voice command.
 15. The displaydevice according to claim 1, wherein the controller is configured todetermine an order in which the content-related information is displayedbased on the requested category.